Automating Data-Model Workflows at a Level-12 HUC Scale in a Distributed Computing Environment

The HydroTerre web services provide the Essential Terrestrial Variable (ETV) datasets to create common hydrological models anywhere in the continental United States (CONUS). These services allow web users to download data for their own purposes in their own computing environment. The datasets are provided using standard Geographic Information System formats and the data transformation is dependent on the users’ own needs, goals, and computing environment. In this article, we demonstrate the feasibility of automating data-transformation workflows for United States Geological Survey level-12 Hydrological Unit Codes (HUC-12) to be consumed in hydrological models. The Penn State Integrated Hydrological Model (PIHM) is demonstrated here, but the workflows serve as a template for other models to adapt and become new services. The focus of this article is the data transformation process, not the model results. We want to demonstrate that workflows empower modelers to create hydrological models rapidly anywhere in the CONUS, and to contribute to a dynamic resource that records provenance of HUC-12 models. To do this, an explanation is required of both the hardware and software architecture because the way in which they are coupled is critical for web service performance. A demonstration of the feasibility to automate data-model workflows for CONUS HUC-12 catchments is discussed with the emphasis on reproducibility by using data-model workflows and distributed computing resources.